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AMENDMENTS TO THE SPECIFICATION 

Please replace the present title with the following amended title: 

Method, Device, and Computer Program for Video Image Processing 

Please amend the paragraph beginning on line 25 of pg. 10 as follows: 

The expression "acquiring a plurality of second interpolated frames which are correlated 
with the plurality of first interpolated frames" is intended to mean acquiring a number of second 
interpolated frames corresponding to the number of first interpolated frames. That is, a pixel 
value within a reference patch is interpolated so that it is assigned at the same pixel position as a 
pixel position in a first interpolated frame that has a pixel value, whereby a second interpolated 
frame corresponding to that first interpolated frame is acquired. And this This processing is 
performed on all of the first interpolated frames. 

Please amend the paragraph beginning on line 7 of pg. 13 as follows: 

Moreover, the coordinate-transformed frame is acquired by transforming the coordinates 
of the image within the second patch of the other frame to a coordinate space of the reference 
frame, based on the correspondent relationship. And th eThe correlation value is computed and 
represents a correlation between the image within the patch of the coordinate-transformed frame 
and the image within the reference patch of the reference frame. The weighting coefficient, 
which is employed when synthesizing the first interpolated frame and the second interpolated 
frame, is computed based on the correlation value. As the correlation between the coordinate- 
transformed frame and the reference frame becomes greater, the weighting coefficient makes the 
weight of the first interpolated frame greater. In the case where three or more frames are 
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sampled, the coordinate-transformed frame, correlation value, and weighting coefficient are 

acquired for each of the frames other than the reference frame. 

Please amend the paragraph beginning on line 17 of pg. 26 as follows: 

The expression "acquiring a plurality of second interpolated frames which are correlated 
with the plurality of first interpolated frames" is intended to mean acquiring a number of second 
interpolated frames corresponding to the number of first interpolated frames. That is, a pixel 
value within a reference patch is interpolated so that it is assigned at the same pixel position as a 
pixel position in a first interpolated frame that has a pixel value, whereby a second interpolated 
frame corresponding to that first interpolated frame is acquired. And this This processing is 
performed on all of the first interpolated frames. 
Please amend the paragraph beginning on line 2 of pg. 38 as follows: 

In a preferred form of the fifth synthesis method of the present invention, the 
aforementioned correspondent relationships are acquired in order of the other frames closer to 
the reference frame, and a correlation is acquired between each of the other frames, in which the 
correspondent relationship is acquired, and the reference frame. And wh e n When the correlation 
is lower than a predetermined threshold value, acquisition of the correspondent relationships is 
stopped, and the synthesized frame is obtained based on the correspondent relationship by 
employing the other frames, in which the correspondent relationship has been acquired, and the 
reference frame. 

Please amend the paragraph beginning on line 1 of pg. 40 as follows: 
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In a preferred form of the fifth video image synthesizer of the present invention, the 
correspondent relationship acquisition means acquires the correspondent relationships in order of 
other frames closer to the reference frame. Also, the fifth video image synthesizer further 
comprises stoppage means for acquiring a correlation between each of the other frames, in which 
the correspondent relationship is acquired by the correspondent relationship acquisition means, 
and the reference frame, and stopping a process which is being performed in the correspondent 
relationship acquisition means when the correlation is lower than a predetermined threshold 
value. And tho The frame synthesis means acquires the synthesized frame by employing the 
other frames, in which the correspondent relationship has been acquired, and the reference frame, 
based on the correspondent relationship acquired by the correspondent relationship acquisition 
means. 

Please amend the paragraph beginning on line 21 of pg. 45 as follows: 

In another preferred form of the sixth video image synthesizer of the present invention, 
the aforementioned correlation computation means computes a histogram for at least one of the 
Y, Cb, and Cr components of each of the adjacent frames (where the Y component is a 
luminance component and the Cb and Cr components are color difference components), also 
computes a Euclidean distance for each component between the adjacent frames by employing 
the histogram, and computes the sum of the Euclidean distances for the three components. And 
when When the sum is a predetermined second threshold value or greater, the aforementioned 
contiguous frame group detection means judges that the correlation between the adjacent frames 
is lower than the predetermined first threshold value. 
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Please amend the paragraph beginning on line 13 of pg. 46 as follows: 

In the sixth video image synthesizer of the present invention, the aforementioned 
correlation computation means may compute a difference between pixel values of corresponding 
pixels of the adjacent frames and also compute the sum of absolute values of the differences for 
all corresponding pixels. And whe n When the sum is a third threshold value or greater, the 
contiguous frame group detection means may judge that the correlation between adjacent frames 
is lower than the predetermined first threshold value. 
Please amend the paragraph beginning on line 19 of pg. 50 as follows: 

In the seventh and eighth synthesis methods of the present invention, when extracting the 
aforementioned important scenes, a correlation between adjacent frames of the video image is 
computed. And a Aset of contiguous frames where the correlation is high can be extracted as 
the frame group that constitutes one or more important scenes. 
Please amend the paragraph beginning on line 7 of pg. 71 as follows: 

Further, a lattice point in the patch PI is moved in the 4 directions along the coordinate 
axes by constant pixel quantities. When this occurs, a rectangular area containing the moved 
lattice point is deformed as shown in Fig. 3, for example. And correlation Correlation values 
between the deformed rectangular area and the corresponding rectangular area of the reference 
patch P0 are computed. These correlation values are assumed to be El (Ax, 0), El (-Ax, 0), E1(0, 
Ay), and El (0, -Ay). 

Please amend the paragraph beginning on line 26 of pg. 72 as follows: 
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Now, consider how a point (x, y) within the patch PI (xn, yn) corresponds to a point (x\ 
y') within the reference patch P0 (x'n, y'n), as illustrated in Fig. 4. First, a point (x, y) within the 
patch PI (xn, yn) is transformed to normalized coordinates (u, v), which are computed by inverse 
transformation of Formulas 5 and 6. And base d Based on the reference patch P0 (x'n, y'n) 
corresponding to the normalized coordinates (u, v), coordinates (x', y') corresponding to the 
point (x, y) are computed by Formulas 5 and 6. The coordinates of a point (x, y) are integer 
coordinates where pixel values are originally present, but there are cases where the coordinates 
of a point (x', y') become real coordinates where no pixel value is present. Therefore, pixel 
values at integer coordinates after transformation are computed as the sum of the weighted pixel 
values of coordinates (x' 5 y'), transformed within an area that is surrounded by 8 neighboring 
integer coordinates adjacent to integer coordinates in the reference patch P0. 
Please amend the paragraph beginning on line 7 of pg. 78 as follows: 

And by By assigning a pixel value in the succeeding frame FrN+1 to all integer 
coordinates of a synthesized image, a pixel value I1N+I(x " , y " ) can be obtained. In this case, 
each pixel value Il(x " , y " ) in the first interpolated frame FrHl becomes I1N+I(x " , y ~ ). 
Please amend the paragraph beginning on line 8 of pg. 84 as follows: ^ 

While the process of acquiring the synthesized frame FrG for the luminance component 
Y has been described, synthesized frames FrG for color difference components Cb and Cr are 
acquired in the same manner. And by By combining a synthesized frame FrG(Y) obtained from 
the luminance component Y and synthesized frames FrG(Cb) and FrG(Cr) obtained from the 
color difference components Cb and Cr, a final synthesized frame is obtained. To expedite 
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processing, it is preferable to estimate a correspondent relationship between the reference frame 
FrN and the succeeding frame FrN+1 only for the luminance component Y, and process the color 
difference components Cb and Cr on the basis of the correspondent relationship estimated for the 
luminance component Y. 

Please amend the paragraph beginning on line 22 of pg. 85 as follows: 

And bas e dB ased on the correspondent relationship estimated by the correspondent 
relationship estimation means 2, the coordinates of the succeeding frame FrN+1 are transformed 
to the coordinate space in the reference frame FrN by the coordinate transformation means 3, 
whereby a coordinate-transformed frame FrTO is acquired (step S4). And th e The correlation 
value d0(x, y) of corresponding pixels of the coordinate-transformed frame FrTO and reference 
frame FrN is computed by the correlation-value computation means 6 (step S5). Further, the 
weight computation means 7 computes a weighting coefficient a( x ° , y ), based on the 
correlation value d0(x, y) (step S6). 

Please amend the paragraph beginning on line 16 of pg. 86 as follows: 

Andinl n the synthesis means 8, a pixel value Il( x ° , y° ) in the first interpolated frame 
FrHl and a pixel value I2( x ° , y° ) in the second interpolated frame FrH2 ar^e synthesized, 
whereby a synthesized frame FrG consisting of a pixel value FrG(x " , y " ) is acquired (step S9), 
and the processing ends. 

Please amend the paragraph beginning on line 25 of pg. 88 as follows: 

And in ln the second embodiment, with weight computation means 7 a weighting 
coefficient a( x ° , y° ) is acquired based on the correlation value d0'(x, y) filtered by the filter 
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means 9, and the weighting coefficient a( x * , y° ) is employed in the weighting and addition 

operations that are performed in the synthesis means 8. 

Please amend the paragraph beginning on line 21 of pg. 91 as follows: 

And byB y performing a weighting operation on corresponding first and second 
interpolated frames Frnit and Frn2t by the weighting coefficient at(x°, y°) and also adding the 
weighted frames, an intermediate synthesized frame Fret with a pixel value Frot(x A , y A ) at 
integer coordinates in a synthesized image is acquired. More specifically, as shown in the 
following Formula 19, the pixel values Ii t (x°, y°) and l2t(x°, y°) of corresponding pixels of the 
first and second interpolated frames Frnit and FrH2t are weighted by employing the 
corresponding weighting coefficient at(x°, y°), and the weighted values are added. In this 
manner, the pixel value FrQt(x A , y A ) of an intermediate synthesized frame Fret is acquired. 
Please amend the paragraph beginning on line 17 of pg. 92 as follows: 

And by By adding the intermediate synthesized frames Fret, a synthesized frame Fro is 
acquired. More specifically, by adding corresponding pixels of intermediate synthesized frames 
Fret with the following Formula 20, a pixel value Fr G (x A , y A ) in a synthesized frame Fr G is 
acquired. 

Please amend the paragraph beginning on line 6 of pg. 93 as follows: 

In the case of acquiring a synthesized frame FrG from three or more frames, first and 
second interpolated frames FrHlt and FrH2t with pixel values at the integer coordinates of a 
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synthesized image, and a weighting coefficient at(x " , y " ) at the integer coordinates, may be 
acquired. In this case, for each frame FrN+t (0 ^ t ^ T-l), pixel values IlN+t( x , y) in each 
frame FrN+t are assigned to all integer coordinates of synthesized coordinates, and a first 
interpolated frame FrHlt with pixel values IlN+t(x " , y " ) (i.e., Ilt(x " , y * )) is acquired. And 
byBy adding the pixel values Ilt(x " , y " ) assigned to all frames FrN+t and the pixel values 
I2t(x " , y " ) of the second interpolated frame FrH2t, a plurality of intermediate synthesized 
frames FrGt are obtained, and they are combined into a synthesized frame FrG. 
Please amend the paragraph beginning on line 19 of pg. 93 as follows: 

More specifically, as shown in the following Formula 21, a pixel value IlN+t(x " , y " ) 
at integer coordinates in a synthesized image is computed for all frames FrN+t. And as As shown 
in Formula 22, an intermediate synthesized frame FrGt is obtained by weighting pixel values 
Ilt(x " , y ~ ) and I2t(x " , y " ), employing a weighting coefficient <x(x ~ , y ~ ). Further, as 
shown in Formula 20, a synthesized frame FrG is acquired by adding the intermediate 
synthesized frames FrGt. 

Please amend the paragraph beginning on line 9 of pg. 97 as follows: 

And in ln synthesis means 8, a pixel value Ii(x°, y°) in the first interpolated frame Frni 
and a pixel value l2(x 0 , y°) in the second interpolated frame Frn2 are synthesized, whereby a 
synthesized frame Fro consisting of a pixel value Fro(x A , y A ) is acquired (step SI 8), and the 
processing ends. 

Please amend the paragraph beginning on line 15 of pg. 106 as follows: 
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And in ln the synthesis means 8, a pixel value Ii(x°, y°) in the first interpolated frame 
Frni and a pixel value l2(x 0 , y°) in the second interpolated frame Frn2 are synthesized, whereby 
a synthesized frame Fr G consisting of a pixel value Fr G (x A , y A ) is acquired (step S60), and the 
processing ends. 

Please amend the paragraph beginning on line 24 of pg. Ill as follows: 

Fig. 24 shows processes that are performed in the fifth embodiment. As with the fourth 
embodiment, consider the case where a first interpolated frame FrHl, a second interpolated 
frame FrH2, and a weighting coefficient a( x ° , y° ) are acquired at real coordinates to which 
pixels of the frame FrN+1 in a synthesized image are assigned. In the video image synthesizer of 
the fifth embodiment, as shown in Fig. 24, video image data MO is first input (step S62). To 
acquire a synthesized frame from the video image data MO, the reduction means 42 of the 
sampling means 1 1 A performs a reduction process on the video image data MO and obtains 
reduced video image data (step S64). The sampling execution means 48 sets a sampling range 
on the basis of a correlation between each reduction frame and a reduction reference frame 
acquired by the correlation acquisition means 44, and samples frames from the video image data 
MO in a range corresponding to the sampling range. The sampling range is from the reduction 
reference frame to a reduction frame, which is closer to the reduction reference frame, between a 
pair of adjacent reduction frames whose correlation is lower than a predetermined threshold 
value. On the other hand, when the processing in the correlation acquisition means 44 is stopped 
by the stoppage means 46, the sampling execution means 48 sets a sampling range from a 
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reduction reference frame to a reduction frame being processed at the time of the stoppage, and 
samples frames from the video image data MO in a range corresponding to the sampling range. 
The S frames sampled by the sampling execution means 48 are output to the correspondent 
relationship acquisition means 12 (step S66). The correspondent relationship acquisition means 
12 places a reference patch on the reference frame FrN (step S68), also places the same patch as 
the reference patch on the succeeding frame FrN+1, and moves and/or deforms the patch until a 
correlation value E between an image within the reference patch and an image within the patch 
of the succeeding frame FrN+1 converges (steps S72 and S74). And the The correspondent 
relationship acquisition means 12 acquires a correspondent relationship between the reference 
frame FrN and the succeeding frame FrN+1 (step S78). The correspondent relationship 
acquisition means 12 performs the processes in steps S72 to S78 on all frames excluding the 
reference frame ("YES" in step S80, step S85). 

Please amend the paragraph beginning on line 5 of pg. 129 as follows: 

Thus, in the eighth embodiment, with respect to frames Fr2 and Fr3 temporally before 
and after frame Frl, similarities b2 and b3 with frame Frl are computed, and if similarities b2 
and b3 are great, contributory degrees (weighting coefficients) 02 and 03 are made greater. And 
feames Frames Fr2 and Fr3 are weighted and added to frame Frl, whereby a processed frame FrG 
is obtained. Because of this, there is no possibility that a frame not similar to frame Frl, as it is, 
will be added to frame Frl . This renders it possible to add frames Fr2 and Fr3 to frame Frl 
while reducing the influence of dissimilar frames. Consequently, a processed frame FrG with 
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high quality can be obtained while reducing blurring that is caused by synthesis of frames whose 
similarity is low. 

Please amend the paragraph beginning on line 7 of pg. 132 as follows: 

Now, a description will be given of operation of the ninth embodiment. Fig. 38 shows 
processes that are performed in the ninth embodiment. First, the sampling means 101 samples 
frames Frl, Fr2, and Fr3 from video image data MO (step S141). Then, in the similarity 
computation means 112, similarities b2(m, n) and b3(m, n) between area Al(m, n) in frame Frl 
and corresponding areas A2(m, n) and A3(m, n) are computed (step SI 42). Next, in the 
contributory degree computation means 113, contributory degrees p2(m, n) and p3(m, n) are 
computed by multiplying similarities b2(m, n) and b3(m, n) by a reference contributory degree k 
(step S143). Andinl n accordance with contributory degrees P2(m, n) and P3(m, n), 
corresponding areas A2(m, n) and A3(m, n) are weighted and added to area Al(m, n), whereby a 
processed frame FrG is obtained (step SI 44) and the processing ends. 
Please amend the paragraph beginning on line 21 of pg. 132 as follows: 

Thus, in the ninth embodiment, frame Frl is partitioned into a plurality of areas Al(m, n), 
and similarities b2(m, n) and b3(m, n) are computed for area A2(m, n) and area A3(m, n) in 
frames Fr2 and Fr3 which correspond to area Al(m, n). And ifl f similarities b2(m, n) and b3(m, 
n) are great, contributory degrees (weighting coefficients) p2(m, n) and P3(m, n) are made 
greater. And corroapondin g Corresponding areas A2(m, n) and area A3(m, n) are weighted and 
added to area Al (m,n), whereby a processed frame FrG is obtained. Because of this, even when 
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a certain area in a video image is moved, blurring can be removed for each area moved. As a 

result, a processed frame FrG with high quality can be obtained. 

Please amend the paragraph beginning on line 6 of pg. 135 as follows: 

As illustrated in Fig. 40, the motion- vector computation means 105 partitions frame Frl 
into m x n block-shaped areas Al(m, n) and moves each of the areas A(m, n) in parallel with 
frame Frl . And wh e n When a correlation between pixel values in area Al (m, n) and frame Fr2 is 
highest, the moved quantity and moving direction of area Al(m, n) is computed as motion vector 
V0(m, n) for that area Al(m, n). Note that when the accumulation of the squares of differences 
between pixel values of area Al(m, n) and frame Fr2 or accumulation of absolute values is 
smallest, the correlation is judged to be highest. 

Please amend the paragraph beginning on line 4 of pg. 138 as follows: 

Next, in the similarity computation means 122, similarities b2(01) and b2(02) between 
subject areas 01, 02 in frame Frl and corresponding subject areas 01(Fr2) and 02(Fr2) in frame 
Fr2 are computed and similarities b3(01) and b3(02) between subject areas 01, 02 in frame Frl 
and corresponding areas 01(Fr3) and 02(Fr3) in frame Fr3 are computed (step SI 55). Next, in 
the contributory degree computation means 123, contributory degrees P2(01),P2(02), P3(01), 
and P3(02) are computed by multiplying similarities b2(01), b2(02) and b3(01), and b3(02) by 
a reference contributory degree k (step SI 56). And inln accordance with contributory degrees 
p2(01) and P2(02) and contributory degrees p3(01) and P3(02), the corresponding subject areas 
01(Fr2) and 02(Fr2) and corresponding subject areas 01(Fr3) and 02(Fr3) are weighted and 
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added to the subject areas 01 and 02, respectively. In this manner, a processed frame FrG is 
obtained (step S 1 57) and the processing ends. 

Please amend the paragraph beginning on line 20 of pg. 138 as follows: 

Thus, in the tenth embodiment, frame Frl is partitioned into a plurality of subject areas 
01 and 02, and similarities b2(01) and b2(02) and similarities b3(01) and b3(02) are 
computed for the subject areas 01(Fr2) and 02(Fr2) and subject areas 01(Fr3) and 02(Fr3) in 
frames Fr2 and Fr3 which correspond to the subject areas 01 and 02. And4flf similarities 
b2(01) and b2(02) and similarities b3(01) and b3(02) are great, contributory degrees 
(weighting coefficients) 02(01), p2(02), (33(01), P3(02) are made greater. Andth eThe 
corresponding subject areas 01(Fr2) and 02(Fr2) and corresponding subject areas 01(Fr3) and 
02(Fr3) are weighted and added to the subject areas 01 and 02, whereby a processed frame FrG 
is obtained. Because of this, even when a certain subject area in a video image is moved, 
blurring can be removed for the subject area moved. As a result, a processed frame FrG with 
higher quality can be obtained. 
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